The Singular Value Decomposition

Carlo Tomasi Any m n matrix of rank r transforms the unit sphere in Rn into an r-dimensional hyperellipsoid in Rm. For instance, the rank-2 matrix A = 1 p2 264 p3 p3 3 3 1 1 375 (1) transforms the unit circle on the plane into an ellipse embedded in three-dimensional space. Figure 1 shows the map y = Ax : Two diametrically opposite points on the unit circle are mapped into the two endpoints of the major axis of the ellipse, and two other diametrically opposite points on the unit circle are mapped into the two endpoints of the minor axis of the ellipse. The lines through these two pairs of points on the unit circle are always orthogonal, as proven in appendix A. This result can be generalized to any m n matrix, but the proof is not trivial. This orthogonality suggests a way to diagonalize a matrix without ever resorting to complex quantities. Consider in fact the map in gure 1, represented by equation (1), and imagine transforming the small box at x on the unit circle into its corresponding point y = Ax (the small box on the ellipse). This transformation can be achieved in three steps (see gure 2): 1. Write x in the frame of reference of the two vectors v1;v2 on the unit circle that map into the major axes of the ellipse. There are a few ways to do this, because axis endpoints come in pairs. Just pick one way, but order v1;v2 so they map into the major